{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## XYZ Pro Features\n",
    "This notebook demonstrates some of the pro features for XYZ Hub API.\n",
    "\n",
    "XYZ paid features can be found here: [xyz pro features](https://www.here.xyz/xyz_pro/).\n",
    "\n",
    "XYZ plans can be found here: [xyz plans](https://developer.here.com/pricing)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Virtual Space\n",
    "A virtual space is described by definition which references other existing spaces(the upstream spaces).\n",
    "Queries being done to a virtual space will return the features of its upstream spaces combined.\n",
    "\n",
    "Below are different predefined operations of how to combine the features of the upstream spaces.\n",
    "\n",
    "- [group](#group_cell)\n",
    "- [merge](#merge_cell)\n",
    "- [override](#override_cell)\n",
    "- [custom](#custom_cell)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Make necessary imports.\n",
    "import os\n",
    "import json\n",
    "import warnings\n",
    "\n",
    "from xyzspaces.datasets import get_chicago_parks_data, get_countries_data\n",
    "from xyzspaces.exceptions import ApiError\n",
    "import xyzspaces"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-warning\">\n",
    "<b>Warning:</b> Before running below cells please make sure you have XYZ Token to interact with xyzspaces. \n",
    "                Please see README.md in notebooks folder for more info on XYZ_TOKEN\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Make a XYZ object\n",
    "try:\n",
    "    xyz_token = os.environ[\"XYZ_TOKEN\"]\n",
    "except KeyError:\n",
    "    xyz_token = \"MY-FANCY-XYZ-TOKEN\"\n",
    "    if xyz_token == \"MY-FANCY-XYZ-TOKEN\":\n",
    "        warnings.warn(\n",
    "            \"Please either set your actual token to env variable XYZ_TOKEN or \"\n",
    "            \"just assign value of your actual token to variable xyz_token above.\"\n",
    "        )\n",
    "xyz = xyzspaces.XYZ(credentials=xyz_token)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create two spaces which will act as upstream spaces for virtual space created later.\n",
    "\n",
    "title1 = \"Testing xyzspaces\"\n",
    "description1 = \"Temporary space containing countries data.\"\n",
    "space1 = xyz.spaces.new(title=title1, description=description1)\n",
    "\n",
    "# Add some data to it space1\n",
    "gj_countries = get_countries_data()\n",
    "space1.add_features(features=gj_countries)\n",
    "space_id1 = space1.info[\"id\"]\n",
    "\n",
    "title2 = \"Testing xyzspaces\"\n",
    "description2 = \"Temporary space containing Chicago parks data.\"\n",
    "space2 = xyz.spaces.new(title=title2, description=description2)\n",
    "\n",
    "# Add some data to space2\n",
    "with open('./data/chicago_parks.geo.json', encoding=\"utf-8-sig\") as json_file:\n",
    "    gj_chicago = json.load(json_file)\n",
    "    \n",
    "space2.add_features(features=gj_chicago)\n",
    "space_id2 = space2.info[\"id\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='group_cell'></a>\n",
    "#### Group\n",
    "Group means to combine the content of the specified spaces. All objects of each space will be part of the response when the virtual space is queried by the user. The information about which object came from which space can be found in the XYZ-namespace in the properties of each feature. When writing back these objects to the virtual space they'll be written back to the upstream space from which they were actually coming."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a new virtual space by grouping two spaces created above.\n",
    "\n",
    "title = \"Virtual Space for coutries and Chicago parks data\"\n",
    "description = \"Test group functionality of virtual space\"\n",
    "\n",
    "upstream_spaces = [space_id1, space_id2]\n",
    "kwargs = {\"virtualspace\": dict(group=upstream_spaces)}\n",
    "\n",
    "vspace = xyz.spaces.virtual(title=title, description=description, **kwargs)\n",
    "print(json.dumps(vspace.info, indent=2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Reading a particular feature from space1 via virtual space.\n",
    "\n",
    "vfeature1 = vspace.get_feature(feature_id=\"FRA\")\n",
    "feature1 = space1.get_feature(feature_id=\"FRA\")\n",
    "assert vfeature1 == feature1\n",
    "\n",
    "# Reading a particular feature from space2 via virtual space.\n",
    "vfeature2 = vspace.get_feature(feature_id=\"LP\")\n",
    "feature2 = space2.get_feature(feature_id=\"LP\")\n",
    "assert vfeature2 == feature2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Deleting a feature from virtual space deletes corresponding feature from upstream space.\n",
    "\n",
    "vspace.delete_feature(feature_id=\"FRA\")\n",
    "try:\n",
    "    space1.get_feature(\"FRA\")\n",
    "except ApiError as err:\n",
    "    print(err)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Delete temporary spaces created.\n",
    "vspace.delete()\n",
    "space1.delete()\n",
    "space2.delete()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='merge_cell'></a>\n",
    "#### Merge\n",
    "Merge means that objects with the same ID will be merged together. If there are duplicate feature-IDs in the various data of the upstream spaces, the duplicates will be merged to build a single feature. The result will be a response that is guaranteed to have no features with duplicate IDs. The merge will happen in the order of the space-references in the specified array. That means objects coming from the second space will overwrite potentially existing property values of objects coming from the first space. The information about which object came from which space(s) can be found in the XYZ-namespace in the properties of each feature. When writing back these objects to the virtual space they'll be written back to the upstream space from which they were actually coming, or the last one in the list if none was specified.When deleting features from the virtual space a new pseudo-deleted feature is written to the last space in the list. Trying to read the feature with that ID from the virtual space is not possible afterward."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create two spaces with duplicate data \n",
    "\n",
    "title1 = \"Testing xyzspaces\"\n",
    "description1 = \"Temporary space containing Chicago parks data.\"\n",
    "space1 = xyz.spaces.new(title=title1, description=description1)\n",
    "\n",
    "with open('./data/chicago_parks.geo.json', encoding=\"utf-8-sig\") as json_file:\n",
    "    gj_chicago = json.load(json_file)\n",
    "    \n",
    "# Add some data to it space1\n",
    "space1.add_features(features=gj_chicago)\n",
    "space_id1 = space1.info[\"id\"]\n",
    "\n",
    "title2 = \"Testing xyzspaces duplicate\"\n",
    "description2 = \"Temporary space containing Chicago parks data duplicate\"\n",
    "space2 = xyz.spaces.new(title=title1, description=description1)\n",
    "\n",
    "# Add some data to it space2\n",
    "space2.add_features(features=gj_chicago)\n",
    "space_id2 = space2.info[\"id\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# update a particular feature of second space so that post merge virtual space will have this feature merged\n",
    "lp = space2.get_feature(\"LP\")\n",
    "space2.update_feature(feature_id=\"LP\", data=lp, add_tags=[\"foo\", \"bar\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a new virtual space by merging two spaces created above.\n",
    "\n",
    "title = \"Virtual Space for coutries and Chicago parks data\"\n",
    "description = \"Test merge functionality of virtual space\"\n",
    "\n",
    "upstream_spaces = [space_id1, space_id2]\n",
    "kwargs = {\"virtualspace\": dict(merge=upstream_spaces)}\n",
    "\n",
    "vspace = xyz.spaces.virtual(title=title, description=description, **kwargs)\n",
    "print(vspace.info)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "vfeature1 = vspace.get_feature(feature_id=\"LP\")\n",
    "assert vfeature1[\"properties\"][\"@ns:com:here:xyz\"][\"tags\"] == [\"foo\", \"bar\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bp = space2.get_feature(\"BP\")\n",
    "space2.update_feature(feature_id=\"BP\", data=lp, add_tags=[\"foo1\", \"bar1\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "vfeature2 = vspace.get_feature(feature_id=\"BP\")\n",
    "assert vfeature2[\"properties\"][\"@ns:com:here:xyz\"][\"tags\"] == [\"foo1\", \"bar1\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "space1.delete()\n",
    "space2.delete()\n",
    "vspace.delete()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='override_cell'></a>\n",
    "#### Override\n",
    "Override means that objects with the same ID will be overridden completely. If there are duplicate feature-IDs in the various data of the upstream spaces, the duplicates will be overridden to result in a single feature. The result will be a response that is guaranteed to have no features with duplicate IDs. The override will happen in the order of the space-references in the specified array. That means objects coming from the second space one will override potentially existing features coming from the first space. The information about which object came from which space can be found in the XYZ-namespace in the properties of each feature. When writing back these objects to the virtual space they'll be written back to the upstream space from which they were actually coming. When deleting features from the virtual space the same rules as for merge apply."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create two spaces with duplicate data \n",
    "\n",
    "title1 = \"Testing xyzspaces\"\n",
    "description1 = \"Temporary space containing Chicago parks data.\"\n",
    "space1 = xyz.spaces.new(title=title1, description=description1)\n",
    "\n",
    "with open('./data/chicago_parks.geo.json', encoding=\"utf-8-sig\") as json_file:\n",
    "    gj_chicago = json.load(json_file)\n",
    "    \n",
    "# Add some data to it space1\n",
    "space1.add_features(features=gj_chicago)\n",
    "space_id1 = space1.info[\"id\"]\n",
    "\n",
    "title2 = \"Testing xyzspaces duplicate\"\n",
    "description2 = \"Temporary space containing Chicago parks data duplicate\"\n",
    "space2 = xyz.spaces.new(title=title1, description=description1)\n",
    "\n",
    "# Add some data to it space2\n",
    "space2.add_features(features=gj_chicago)\n",
    "space_id2 = space2.info[\"id\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a new virtual space by override operation.\n",
    "\n",
    "title = \"Virtual Space for coutries and Chicago parks data\"\n",
    "description = \"Test merge functionality of virtual space\"\n",
    "\n",
    "upstream_spaces = [space_id1, space_id2]\n",
    "kwargs = {\"virtualspace\": dict(override=upstream_spaces)}\n",
    "\n",
    "vspace = xyz.spaces.virtual(title=title, description=description, **kwargs)\n",
    "print(vspace.info)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bp = space2.get_feature(\"BP\")\n",
    "space2.update_feature(feature_id=\"BP\", data=bp, add_tags=[\"foo1\", \"bar1\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "vfeature2 = vspace.get_feature(feature_id=\"BP\")\n",
    "assert vfeature2[\"properties\"][\"@ns:com:here:xyz\"][\"tags\"] == [\"foo1\", \"bar1\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "space1.delete()\n",
    "space2.delete()\n",
    "vspace.delete()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Applying clustering in space"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create two spaces which will act as upstream spaces for virtual space created later.\n",
    "\n",
    "title1 = \"Testing xyzspaces\"\n",
    "description1 = \"Temporary space containing countries data.\"\n",
    "space1 = xyz.spaces.new(title=title1, description=description1)\n",
    "\n",
    "# Add some data to it space1\n",
    "gj_countries = get_countries_data()\n",
    "space1.add_features(features=gj_countries)\n",
    "space_id1 = space1.info[\"id\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Genereate clustering for the space\n",
    "space1.cluster(clustering='hexbin')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Delete created space\n",
    "space1.delete()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Rule based Tagging\n",
    "Rule based tagging makes tagging multiple features in space tagged to a particular tag, based in rules mentioned based on JSON-path expression. Users can update space with a map of rules where the key is the tag to be applied to all features matching the JSON-path expression being the value.\n",
    "\n",
    "If multiple rules are matching, multiple tags will be applied to the according to matched sets of features. It could even happen that a feature is matched by multiple rules and thus multiple tags will get added to it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a new space\n",
    "title = \"Testing xyzspaces\"\n",
    "description = \"Temporary space containing Chicago parks data.\"\n",
    "space = xyz.spaces.new(title=title, description=description)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Add data to the space.\n",
    "with open('./data/chicago_parks.geo.json', encoding=\"utf-8-sig\") as json_file:\n",
    "    gj_chicago = json.load(json_file)\n",
    "_ = space.add_features(features=gj_chicago)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# update space to add tagging rules to the above mentioned space.\n",
    "tagging_rules = {\n",
    "        \"large\": \"$.features[?(@.properties.area>=500)]\",\n",
    "        \"small\": \"$.features[?(@.properties.area<500)]\",\n",
    "    }\n",
    "_ = space.update(tagging_rules=tagging_rules)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# verify that features are tagged correctly based on rules.\n",
    "large_parks = space.search(tags=[\"large\"])\n",
    "for park in large_parks:\n",
    "    assert park[\"id\"] in [\"LP\", \"BP\", \"JP\"]\n",
    "small_parks = space.search(tags=[\"small\"])\n",
    "for park in small_parks:\n",
    "    assert park[\"id\"] in [\"MP\", \"GP\", \"HP\", \"DP\", \"CP\", \"COP\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Delete created space\n",
    "space.delete()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Activity Log\n",
    "The Activity log  will enable tracking of changes in your space.\n",
    "To activate it, just create a space with the listener added and enable_uuid set to True.\n",
    "More information on the activity log can be found [here](https://www.here.xyz/api/devguide/activitylogguide/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "title = \"Activity-Log Test\"\n",
    "description = \"Activity-Log Test\"\n",
    "listeners = {\n",
    "    \"id\": \"activity-log\",\n",
    "    \"params\": {\"states\": 5, \"storageMode\": \"DIFF_ONLY\", \"writeInvalidatedAt\": \"true\"},\n",
    "    \"eventTypes\": [\"ModifySpaceEvent.request\"],\n",
    "}\n",
    "space = xyz.spaces.new(\n",
    "    title=title, description=description, enable_uuid=True, listeners=listeners,\n",
    ")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from time import sleep\n",
    "\n",
    "# As activity log is async operation adding sleep to get info\n",
    "sleep(5)\n",
    "print(json.dumps(space.info, indent=2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "space.delete()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
